1. Field of the Invention
The present invention relates to image processing for changing a focus, depth of field, and the like of a captured photo image.
2. Description of the Related Art
A method of changing a focus, depth of field, and the like of a captured photo image has been proposed. For example, R. Ng, M. Levoy, et al. “Light Field Photography with a Hand-Held Plenoptic Camera” Stanford University Computer Science Tech Report CSTR 2005-02, April 2005 discloses an arrangement of a plenoptic camera which records a light ray state inside a capturing optical system, and a development method for changing a focus of (refocusing) a captured photo image. Also, A. Isaksen, et al. “Dynamically Reparameterized Light Fields” ACM SIGGRAPH, pp. 297-306 (2000) discloses a method of generating an image, a depth of field of which is changed (to be referred to as depth of field control hereinafter) from images of a multiple camera having a plurality of capturing devices, which are compact and have a deep depth of field.
In general, in a multiple camera prepared by laying out a plurality of compact cameras, each camera has a small number of pixels and a low resolution due to a compact camera. The plenoptic camera has a lower spatial resolution than a normal camera having the same number of pixels as the plenoptic camera.
T. Georgiev and A. Lumsdaine “Superresolution with Plenoptic 2.0 cameras” Adobe Technical Report (2009) discloses a method of increasing a resolution by executing super resolution processing when data acquired by a plenoptic camera are developed. In case of a multiple camera, a plurality of low-resolution images including an object having slight position shifts are obtained. A technique for obtaining one high-resolution image from these low-resolution images by super resolution processing is known (for example, Sung C. P. and Min K. P. “Super-Resolution Image Reconstruction: A Technical Overview” IEEE Signal Proc. Magazine, Vol. 26, No. 3, p. 21-36 (2003)).
In order to execute the super resolution processing in the multiple camera, a plurality of images having position shifts, which are not integer multiples of a pixel size, or pieces of information equivalent to them are required. However, the multiple camera attains refocusing and depth of field control by shifting images of individual cameras and superposing these images. For this reason, no position shift is generated depending on a distance to be focused and depth of field. In such case, even when the super resolution processing is executed, not only a satisfactory high-resolution image cannot be obtained, but also an image having low image quality, which is hard to be appreciated, is generated due to generation of ringing and the like.